Aligning Eyes between Humans and Deep Neural Network through Interactive Attention Alignment

While Deep Neural Networks (DNNs) are deriving the major innovations through their powerful automation, we also witnessing peril behind automation as a form of bias, such automated racism, gender and adversarial bias. As societal impact DNNs grows, finding an effective way to steer align behavior with human mental model has become indispensable in realizing fair accountable models. establishing adjust "think like humans'' is pressing need, there have been few approaches aiming capture how "humans would think'' when introduce biased reasoning seeing new instance. We propose Interactive Attention Alignment (IAA), framework that uses methods for visualizing attention, saliency maps, interactive medium humans can leverage unveil cases DNN's directly attention. To realize more human-steerable than state-of-the-art, IAA introduces two novel devices. First, Reasonability Matrix systematically identify Second, applies GRADIA, computational pipeline designed effectively applying adjusted attention jointly maximize quality prediction accuracy. evaluated Study 1 GRADIA 2 classification problem. In 1, found bias detection significantly improve perceived from eyes not Matrix. 2, using improves (1) human-assessed (2) performance scenarios where training samples limited. Based on our observation studies, present implications future design problem space social computing data annotation toward achieving human-centered steerable AI.